A Simple Nearly Optimal Restart Scheme For Speeding Up First-Order Methods

We present a simple scheme for restarting first-order methods convex optimization problems. Restarts are made based only on achieving specified decreases in objective values, the amounts being same all Unlike existing restart schemes, makes no attempt to learn parameter values characterizing structure of an problem, nor does it require any special information that would not be available practice (unless method chosen employed itself requires information). As immediate corollaries main theorems, we show when some well-known scheme, resulting complexity bounds nearly optimal particular—yet quite general—classes